{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "de765f16-341b-4dc3-b59d-b16437c7e050",
   "metadata": {},
   "source": [
    "\n",
    "# Ollama Chat\n",
    "\n",
    "Please refer to langchain guideline for [Retrieval](https://python.langchain.com/docs/modules/data_connection/)\n",
    "\n",
    "Vector store-backed retriever\n",
    "\n",
    "https://python.langchain.com/docs/modules/data_connection/retrievers/vectorstore\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "e25cdbdc-daee-4a9b-b363-9080dc216927",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Name: langchain\n",
      "Version: 0.1.9\n",
      "Summary: Building applications with LLMs through composability\n",
      "Home-page: https://github.com/langchain-ai/langchain\n",
      "Author: \n",
      "Author-email: \n",
      "License: MIT\n",
      "Location: /opt/conda/lib/python3.11/site-packages\n",
      "Requires: aiohttp, dataclasses-json, jsonpatch, langchain-community, langchain-core, langsmith, numpy, pydantic, PyYAML, requests, SQLAlchemy, tenacity\n",
      "Required-by: \n"
     ]
    }
   ],
   "source": [
    "!pip show langchain"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5357c803-4015-428e-a8b5-870d4abcab01",
   "metadata": {},
   "source": [
    "## Load instruction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "c2f43f97-861d-4933-a83b-02b8515a186e",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 1/1 [00:00<00:00, 285.58it/s]\n"
     ]
    }
   ],
   "source": [
    "from langchain_community.document_loaders import DirectoryLoader, TextLoader\n",
    "\n",
    "loader = DirectoryLoader('structurizr/llm', glob=\"instruction.txt\", show_progress=True, use_multithreading=True, loader_cls=TextLoader)\n",
    "\n",
    "instruction = loader.load()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "cae9272a-3606-41fe-aae0-895ac2aea4af",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "You are an expert on structurizr which is a domain specific language (DSL) for software architecture modeling and documenation.\n",
      "You will be provided a keyword in structurizr DSL, you need to provide the keyword gramma precisely and consistently.\n",
      "You need to check the permited children of the keyword.\n",
      "You must strictly adhere to the keyword gramma to generate the output.\n",
      "You need to embed the permited children of the keyword in the gramma.\n",
      "You do not need to provide the permited children out of the gramma.\n",
      "You need to provide the gramma, you do not need to provide any explanations.\n",
      "You do not need to privde code examples or explanations. \n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(instruction[0].page_content)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6ae76f53-0e58-499c-91f1-fd9de4175119",
   "metadata": {},
   "source": [
    "## Retrieval"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "e07b2c7d-62d7-457c-bbcb-298b29aa522c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Please define the following env variables in the .env file \n",
    "# \n",
    "# ollama_url = http://ollama:11434\n",
    "# model_name = mistral:instruct\n",
    "# embedding_model=nomic-embed-text\n",
    "# num_thread=8\n",
    "# "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "fa2048ea-dbc4-49d7-b855-8b546dce2edb",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Use the utility.py to get llm, embedding and output parser\n",
    "import utility\n",
    "embeddings = utility.get_embeddings()\n",
    "llm = utility.get_llm()\n",
    "output_parser = utility.get_output_parser()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "b884ed11-1e88-4ccf-9111-787c0b606522",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 249 ms, sys: 33.4 ms, total: 282 ms\n",
      "Wall time: 631 ms\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "import pyarrow as pa\n",
    "import lancedb\n",
    "from langchain_community.vectorstores import LanceDB\n",
    "\n",
    "db = lancedb.connect(\"lancedb\")\n",
    "\n",
    "language = db.open_table('language')\n",
    "\n",
    "language_vectorstore = LanceDB(language, embeddings)\n",
    "\n",
    "retriever = language_vectorstore.as_retriever(search_kwargs={\"k\": 1})"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ffccd7ce-cea2-495a-a565-7bc944e0d639",
   "metadata": {},
   "source": [
    "## ChatMessage"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "9ca6624d-8d97-4f95-baff-0548717c6894",
   "metadata": {},
   "outputs": [],
   "source": [
    "# keyword = \"workspace\"\n",
    "\n",
    "# docs = retriever.get_relevant_documents(keyword)\n",
    "\n",
    "# language_gramma = f\"The gramma of {keyword} is provided here: {docs[0].page_content}\" "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "dd527cf0-379d-4800-bff7-02b7cfdf41c3",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_core.prompts import ChatPromptTemplate\n",
    "\n",
    "prompt = ChatPromptTemplate.from_messages([\n",
    "    (\"system\", instruction[0].page_content),\n",
    "    (\"system\", \"please answer the question in following context:\\n{context}\"),\n",
    "    (\"user\", \"{input}\")\n",
    "])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "081c8e88-e45c-4b32-bc0e-206801565f83",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_core.runnables import RunnableParallel, RunnablePassthrough\n",
    "\n",
    "retrieval = RunnableParallel(\n",
    "    {\"context\": retriever, \"input\": RunnablePassthrough()}\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "923b1ad3-1926-4713-be02-b378f9914b21",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "**************************\n",
      "The `container` keyword defines a container, within a software system.  \n",
      "```\n",
      "container <name> [description] [technology] [tags] {\n",
      "...\n",
      "}\n",
      "```  \n",
      "The following tags are added by default:  \n",
      "- `Element`\n",
      "- `Container`  \n",
      "Permitted children:  \n",
      "- [!docs](#documentation)\n",
      "- [!adrs](#architecture-decision-records-adrs)\n",
      "- [group](#group)\n",
      "- [component](#component)\n",
      "- [description](#description)\n",
      "- [technology](#technology)\n",
      "- [tags](#tags)\n",
      "- [url](#url)\n",
      "- [properties](#properties)\n",
      "- [perspectives](#perspectives)\n",
      "- [-> (relationship)](#relationship)\n"
     ]
    }
   ],
   "source": [
    "# debug the context from retriever\n",
    "for doc in retriever.invoke(\"what is the grammar of container\"):\n",
    "    print(f'**************************\\n{doc.page_content}', flush=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "676b7ce8-8bca-4ff6-8eee-3705047c90ad",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "You are an expert on structurizr which is a domain specific language (DSL) for software architecture modeling and documenation.\n",
      "You will be provided a keyword in structurizr DSL, you need to provide the keyword gramma precisely and consistently.\n",
      "You need to check the permited children of the keyword.\n",
      "You must strictly adhere to the keyword gramma to generate the output.\n",
      "You need to embed the permited children of the keyword in the gramma.\n",
      "You do not need to provide the permited children out of the gramma.\n",
      "You need to provide the gramma, you do not need to provide any explanations.\n",
      "You do not need to privde code examples or explanations. \n",
      "\n",
      "please answer the question in following context:\n",
      "[Document(page_content='`workspace` is the top level language construct, and the wrapper for the [model](#model) and [views](#views). A workspace can optionally be given a name and description.  \\n```\\nworkspace [name] [description] {\\n...\\n}\\n```  \\nA workspace can also extend another workspace, to add more elements, relationships, views, etc to it.  \\n```\\nworkspace extends <file|url> {\\n...\\n}\\n```  \\nThe base workspace can either be referenced using a local DSL/JSON file, or a remote (via a HTTPS URL) DSL/JSON file.\\nWhen extending a DSL-based workspace, all of the identifiers defined in that workspace are available to use in the extended workspace.  \\nPermitted children:  \\n- name <name>\\n- description <description>\\n- [properties](#properties)\\n- [!docs](#documentation)\\n- [!adrs](#architecture-decision-records-adrs)\\n- [!identifiers](#identifier-scope)\\n- [!impliedRelationships](#impliedrelationships)\\n- [model](#model)\\n- [views](#views)\\n- [configuration](#configuration)', metadata={'vector': [-0.3367006182670593, 1.2278048992156982, -2.112365961074829, -1.9728574752807617, 0.7058570981025696, -0.9843620657920837, 0.026085492223501205, 0.3938854932785034, -0.231288343667984, -0.9752561450004578, -0.5551283955574036, 0.6621705293655396, 2.3993306159973145, 0.33369138836860657, 0.2145831286907196, -0.5312903523445129, -0.2813478112220764, -0.5297631025314331, 0.21259911358356476, -0.788287878036499, -0.18844151496887207, -0.9259121417999268, -0.565975546836853, -0.30752310156822205, 1.570773720741272, 0.7944910526275635, -0.033112820237874985, -0.02603111043572426, -0.32024842500686646, 0.11862557381391525, 0.342608779668808, 0.08456483483314514, 0.5716313123703003, 0.01584353856742382, -0.8633930683135986, -1.231186866760254, 0.34324565529823303, 0.7701078057289124, -1.383244514465332, 0.33117786049842834, 0.15099765360355377, -0.40626490116119385, -0.06096888706088066, -1.3314510583877563, 0.03238961845636368, -0.7728101015090942, 0.21849650144577026, -1.0478403568267822, 0.8518261313438416, -0.7833083271980286, -0.4201163053512573, -0.2101636677980423, 0.07819338887929916, -0.057088159024715424, 1.7119050025939941, 0.1892501413822174, -0.5750680565834045, 0.4822505712509155, -0.16687780618667603, -0.4560264050960541, 1.2092630863189697, 2.0215392112731934, -0.2648670971393585, 1.4741261005401611, 0.6891084909439087, 0.7842545509338379, -1.113184928894043, 0.5277273654937744, -0.31389155983924866, -1.2887463569641113, 0.8526769280433655, 0.6378214955329895, 0.1437039077281952, -0.5964949727058411, -0.9651527404785156, 0.8437082171440125, 0.1437482237815857, -0.4200803339481354, -0.37055468559265137, 0.8156977295875549, -0.6957245469093323, 0.4879165589809418, 0.8247957229614258, -0.865371823310852, 1.1859409809112549, 0.7592555284500122, -0.1710352599620819, 0.023159021511673927, -0.10780738294124603, 1.6964863538742065, 0.4368153214454651, -0.5411785244941711, 0.3460215628147125, 0.41555312275886536, -1.4881867170333862, -0.07437116652727127, 0.03157360479235649, 0.1721964031457901, -0.2820557653903961, -0.6814295053482056, -0.5198372006416321, -0.6527621746063232, 0.948802649974823, 0.03341073915362358, 0.11204300820827484, 1.2726757526397705, -0.39417585730552673, 0.21327774226665497, 0.15379205346107483, -0.45569032430648804, -0.1067129448056221, 0.45713165402412415, -0.44171154499053955, -0.48799562454223633, 0.016486160457134247, -1.0558643341064453, 1.0491864681243896, -0.6140947937965393, -0.37068119645118713, 0.6544051170349121, -0.3183073103427887, 0.563012957572937, -0.009177306666970253, 0.710185706615448, 0.6872475147247314, 0.24706344306468964, -0.7709643244743347, -0.9625343680381775, 0.6527290940284729, -0.4367077648639679, -0.16061536967754364, -0.35950279235839844, -0.682344377040863, 0.5976106524467468, 0.15863537788391113, 0.6282382607460022, -0.3898770213127136, -0.6778547167778015, -0.3715984523296356, 0.4620487093925476, -0.13925884664058685, 0.8972252607345581, -0.4240851402282715, -0.5851973295211792, 0.38539913296699524, -1.0009180307388306, 0.9440548419952393, -0.05917004495859146, 0.23201173543930054, 0.656200647354126, -0.6069929599761963, 0.9945002794265747, -1.0336251258850098, 0.4890798032283783, 0.649980366230011, -0.6917913556098938, 0.33836352825164795, -0.1963425576686859, 0.6337765455245972, 0.7491371631622314, 1.0446140766143799, 0.9499197602272034, -1.0490530729293823, 1.0408082008361816, -0.5004414916038513, -1.275759220123291, 0.874566912651062, 0.42438268661499023, 0.2922664284706116, 0.33657586574554443, -0.27614152431488037, -0.8048730492591858, -0.04948223754763603, -0.15749338269233704, 0.09377798438072205, 0.030154520645737648, 1.0683174133300781, -1.1606632471084595, -0.533475935459137, -0.5438038110733032, 0.6034690737724304, -1.2038781642913818, 0.48711320757865906, 0.5952664613723755, 0.353060245513916, 0.924433171749115, -0.012288931757211685, -1.112205147743225, -0.05039103701710701, -0.9413174986839294, -0.7103923559188843, 0.3760039508342743, -0.20609109103679657, 0.3317583203315735, -0.8788817524909973, -1.136189579963684, 0.31316429376602173, -0.12478715926408768, -0.1275012195110321, -0.023179389536380768, -0.30671241879463196, 0.12837330996990204, -0.9296752214431763, 1.2703098058700562, -0.7307207584381104, 0.3154323697090149, -0.9922328591346741, 0.335085928440094, 0.185186505317688, 0.62134850025177, 1.8139152526855469, -0.28478333353996277, -0.6657425165176392, -0.06523066014051437, 0.5084587931632996, -0.6367661356925964, -0.24942253530025482, -0.21301862597465515, -0.5619716644287109, -0.21739953756332397, -0.11949378252029419, -0.5329355001449585, 0.9508681893348694, 0.42131370306015015, 0.6913177371025085, -0.5023329257965088, -0.6737396121025085, 0.4202648401260376, -0.5962669253349304, 0.25263121724128723, -1.325942039489746, -0.9539095759391785, 0.32606738805770874, -0.8188132047653198, -0.2594841718673706, 0.46497049927711487, 0.33229508996009827, 0.6612261533737183, 0.1525098830461502, 0.13182386755943298, 0.3281456530094147, -0.061887226998806, -0.5933127999305725, -0.1174294725060463, -1.1236255168914795, 0.7128162384033203, -0.03945128619670868, -0.3307018578052521, 0.6655913591384888, 1.0263328552246094, -0.6829705238342285, -0.27970319986343384, 0.31477028131484985, 0.16058461368083954, 0.21226118505001068, -1.0741336345672607, -0.2630060911178589, -0.3759476840496063, -0.16134369373321533, -0.9323667287826538, 0.304111123085022, -0.6455680727958679, 0.7826105952262878, -0.19250167906284332, -0.38801953196525574, -0.6518416404724121, 0.5179296731948853, -0.1631096601486206, 0.8881076574325562, -0.7847036719322205, 0.9034410119056702, 0.2598922550678253, -0.667485237121582, 0.5489475727081299, -0.15865835547447205, 0.6590776443481445, -0.3029339611530304, -0.09585663676261902, -0.5452804565429688, 0.14332795143127441, -0.24576888978481293, -0.30854886770248413, -0.17088286578655243, -0.5019630193710327, 0.6468048095703125, 0.5690564513206482, -0.03869986906647682, -0.1560218781232834, 0.27727678418159485, -0.7634984254837036, 0.8243139386177063, 0.5036463737487793, -1.0186591148376465, 0.6644127368927002, 0.027251534163951874, -0.04012920707464218, 0.88032466173172, -0.07593114674091339, -0.017634237185120583, -0.33773550391197205, 0.6906982064247131, 1.170836329460144, 0.6906049251556396, 1.2638636827468872, -0.0031264801509678364, -0.8784834742546082, -0.2939242720603943, 0.5120510458946228, 0.04211759939789772, 0.38377684354782104, -0.39345070719718933, -0.3873101472854614, -0.7297289967536926, 0.23667018115520477, -0.680204451084137, 0.9916102290153503, 0.19853045046329498, 0.1832202970981598, 1.0483589172363281, -0.5966649055480957, -0.2582244873046875, -0.63815838098526, -0.19133980572223663, -0.5254195332527161, 0.5488855838775635, 0.18068186938762665, -0.5382696390151978, -0.10025382786989212, -0.4493943750858307, 0.3728162348270416, 0.3218366205692291, -0.26416561007499695, 0.7689904570579529, -0.6775138974189758, -0.6538372039794922, 0.07184797525405884, -0.16085723042488098, -0.09956847876310349, 0.7188088893890381, 0.8476488590240479, 1.2134004831314087, -0.5483765006065369, 0.1644858717918396, -0.13364475965499878, -0.06557242572307587, 0.05244589224457741, -0.7322260737419128, -0.362996906042099, 1.1904276609420776, 0.8591991662979126, -1.1153615713119507, 0.27445387840270996, -0.4947052299976349, 0.09890629351139069, -0.2080918550491333, -0.05663347989320755, 0.8270089626312256, 0.885172426700592, 0.12665648758411407, -0.3393995463848114, 0.5849983096122742, -0.4339640736579895, -0.12324835360050201, -1.254658579826355, 0.5852439999580383, 0.6774754524230957, 1.5826815366744995, 0.26979199051856995, 0.27681270241737366, 0.31688833236694336, 0.34845513105392456, -0.6037394404411316, -0.13933172821998596, 0.6082769632339478, 0.7141236066818237, -0.43407297134399414, -0.9540348649024963, -0.06378434598445892, -0.7165022492408752, -0.10334120690822601, 0.48288899660110474, 0.021393390372395515, 0.08122557401657104, 0.7764209508895874, 0.198629692196846, 0.28345155715942383, -0.08576563000679016, 0.13769377768039703, -0.028633244335651398, 0.13015951216220856, -0.3900812268257141, -0.21153132617473602, -0.49853506684303284, -0.4603061378002167, 0.37672290205955505, -0.41586244106292725, 0.08680986613035202, 0.5042380094528198, -0.09915515035390854, 0.3984903395175934, -0.7229204773902893, -0.6460831165313721, 0.2612278163433075, -0.23122026026248932, 0.4819875955581665, 0.21891601383686066, -0.0632455050945282, -0.17268259823322296, 0.6206010580062866, -0.30965468287467957, 0.5416079163551331, 1.1343287229537964, -0.12452421337366104, -0.9804446697235107, 0.3916241228580475, -0.9025916457176208, 0.45237088203430176, -0.1711345911026001, -0.7681756615638733, 0.4758908748626709, 0.19476722180843353, 0.3740658462047577, 0.24319416284561157, 0.08692318946123123, 1.310978651046753, -0.19087617099285126, 0.4341834783554077, 1.0381677150726318, 0.2361825853586197, -1.2516345977783203, -0.7142848372459412, 0.9108123779296875, 0.7823230028152466, 0.1893194019794464, -0.9152236580848694, 0.6075412034988403, -0.34900420904159546, 0.6791881918907166, -0.6610547304153442, 1.132538914680481, 0.4292915463447571, -0.6006208062171936, 0.06994522362947464, -0.23997263610363007, 0.11156866699457169, 1.4140403270721436, 0.11881516873836517, -0.9865322113037109, -1.2322028875350952, 0.6778845191001892, -0.04960399493575096, 0.9601366519927979, 0.12656456232070923, 0.12933112680912018, 2.2407307624816895, -0.5466830730438232, -0.413527250289917, -0.2768252491950989, 0.469069242477417, -0.06867487728595734, -0.11549513041973114, 0.5395075678825378, -0.7369396686553955, 0.13792464137077332, 0.47607436776161194, -0.06515752524137497, -0.42858394980430603, -0.6536766886711121, 0.5544399619102478, 1.3435701131820679, 0.18065115809440613, -0.4435864984989166, 0.13905292749404907, -0.8335209488868713, 0.427463561296463, -0.01821955293416977, 0.43715816736221313, 0.6267189979553223, -0.19644799828529358, 0.5251560807228088, -0.8310320377349854, 0.3620675802230835, -0.9437296390533447, -0.8113115429878235, 0.3998969793319702, 0.5962125658988953, -0.08420202881097794, -0.57695072889328, -0.08525992184877396, 0.03659139573574066, 0.6586400270462036, 0.4532737135887146, -0.03628167510032654, 1.5121127367019653, -0.922744870185852, -0.031445227563381195, -0.42395663261413574, 0.18543308973312378, -0.05684487149119377, 0.22884680330753326, 1.0923569202423096, 0.48983755707740784, 0.5898116230964661, 0.81036776304245, -0.17718736827373505, -0.8307814598083496, 0.7169293761253357, -0.7997084856033325, -1.1256719827651978, -1.3618370294570923, 0.026212234050035477, 0.33762094378471375, -0.20601041615009308, 0.6045347452163696, 1.0061274766921997, -1.1683212518692017, -0.6458456516265869, 0.05795932933688164, -0.9733226895332336, 0.02346726320683956, 0.1469453126192093, -1.29682457447052, 0.4192703664302826, -0.9976146221160889, -1.1803661584854126, 1.1405552625656128, 0.27714526653289795, -0.5740044116973877, 0.37054774165153503, 0.6915559768676758, 0.5335045456886292, 0.6463086605072021, -0.4450267553329468, 0.2882988750934601, 0.7415408492088318, -0.7954014539718628, -1.4197204113006592, 0.5683300495147705, 0.05397609993815422, 0.03892434760928154, -0.2998378872871399, -0.2845939099788666, -0.36284518241882324, -0.41774722933769226, -0.5770949721336365, -0.3853459656238556, -0.29094916582107544, 0.4868842661380768, 0.3012200891971588, -0.9273547530174255, 0.29685014486312866, -1.3446282148361206, 0.2374153733253479, -0.8523753881454468, 0.8888260126113892, -1.2057350873947144, 0.7523120641708374, -0.26992326974868774, 0.45317837595939636, -0.4336203634738922, 0.11409713327884674, 0.7569641470909119, 0.41097012162208557, 0.4903218746185303, -0.97042316198349, 0.22618155181407928, 0.6581432223320007, -0.08282448351383209, -0.11413247138261795, 1.1108906269073486, 0.11752190440893173, -0.6894633173942566, 0.3611258566379547, 0.08975137770175934, -0.5922926068305969, 0.33476030826568604, 0.755211591720581, -0.2620964050292969, -1.0192923545837402, -0.9818783402442932, -0.22123859822750092, -0.6266299486160278, 0.4633949398994446, -0.5869379639625549, -1.1052576303482056, 0.6410539746284485, -0.48307138681411743, -0.22564852237701416, 0.4250914752483368, -1.0970135927200317, 0.15461398661136627, -0.8269080519676208, 0.8635252118110657, 0.34922221302986145, -0.25312334299087524, 0.4076528549194336, 0.29008984565734863, -0.30968043208122253, 0.19094912707805634, -1.0324103832244873, -0.28983548283576965, 0.6554927825927734, 0.5653368234634399, -0.4948021173477173, 0.713750422000885, 0.5113646388053894, -0.9721914529800415, -0.09601705521345139, 0.15194137394428253, -0.2409432828426361, 0.291204035282135, 0.08812408894300461, 0.18676543235778809, 0.7462406158447266, 1.0181821584701538, -0.3099394738674164, 0.2340196967124939, -0.33959004282951355, 0.38599535822868347, -0.3377949595451355, -0.6896225810050964, -1.3522833585739136, 0.6136596202850342, 0.02643662877380848, 0.10995058715343475, 0.4677075743675232, -1.03464937210083, -0.09359724819660187, -0.3469903767108917, 2.1890487670898438, -1.2798664569854736, 0.825534462928772, -1.368397831916809, -1.2444953918457031, -1.8088722229003906, 0.32032644748687744, -0.18129917979240417, -0.05670265853404999, 0.018570495769381523, 0.6716850399971008, 0.6877224445343018, -0.47368666529655457, 0.2316894382238388, 0.3913774788379669, 0.074130117893219, -1.0380651950836182, 0.8455272316932678, 0.29951179027557373, 0.44667142629623413, -0.17355704307556152, 1.9317564964294434, 1.3467512130737305, 0.1560761034488678, 0.46046918630599976, -0.020872360095381737, -0.4430123567581177, 0.47270992398262024, -0.9123899936676025, -1.344541311264038, 0.39359554648399353, 0.03461732342839241, 0.074502132833004, -0.5889225006103516, -0.39849182963371277, 0.490336149930954, 0.07989828288555145, 0.2861008048057556, 0.9119997620582581, -1.0091897249221802, -0.4579299986362457, 0.42552417516708374, 0.10140134394168854, 0.5025582313537598, 0.6983601450920105, 0.0649091899394989, 0.5735647678375244, 0.7005746960639954, 0.9210378527641296, 0.1999170333147049, -0.7702190279960632, 0.47088807821273804, 0.5863855481147766, 0.50691157579422, 0.16944997012615204, -0.4440843462944031, -1.8733270168304443, -0.18401873111724854, -0.35695183277130127, -0.4195825159549713, -0.0994340032339096, -0.8620306253433228, -0.7085175514221191, 1.1929875612258911, -0.6194138526916504, -0.46039921045303345, 0.16510255634784698, 0.8274635672569275, -0.45603036880493164, -0.3969375789165497, 1.3685225248336792, -0.08666238933801651, 0.16118483245372772, -0.02087322250008583, 0.5933021903038025, -0.31219419836997986, 0.23491019010543823, -0.037453170865774155, 0.27429282665252686, -0.3336316645145416, -0.8412371277809143, -0.3037071228027344, 0.9278504848480225, -0.2117251306772232, 0.39157989621162415, 0.19420254230499268, 0.7972860932350159, -0.0345768928527832, -0.43267083168029785, 0.1256203055381775, 0.04268502816557884, 0.6245540380477905, -0.15467628836631775, -0.7755571603775024, -0.4335850179195404, 0.03914282098412514, -0.42413920164108276, 0.34789228439331055, -0.9203366637229919, 0.4343831241130829, 0.9631713628768921, -0.6619488596916199, -0.023344960063695908, -0.4423201084136963, 1.5231460332870483, -0.9946919083595276, 0.7638593316078186, -0.42985546588897705, -0.7023813724517822, -0.7648715376853943, -0.2787427008152008, -0.3287091553211212, -0.4728955626487732, -0.07113252580165863, 0.6156468391418457, -0.6274003982543945, 0.45945799350738525, -0.7625044584274292, 0.274333655834198, 0.8682374358177185, 0.2192758172750473, 0.23985058069229126, -0.1764305979013443, -0.41148197650909424, -0.19243532419204712, 0.32216712832450867, -0.41912320256233215, 0.18753033876419067, 0.887019157409668, 1.6789131164550781, 0.8708255290985107, 0.27678993344306946, 0.33362266421318054, 0.5389842391014099, -0.2981942296028137, -0.3140742778778076, -1.622700572013855, -0.9923866987228394, -0.2605375051498413], '_distance': 315.1009826660156})]\n",
      "what is the grammar of workspace\n"
     ]
    }
   ],
   "source": [
    "## debug the chat message \n",
    "for chat_message in (retrieval | prompt).invoke(\"what is the grammar of workspace\").messages:\n",
    "    print(chat_message.content, flush=True)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "54f327d2-e487-4149-a510-b2564aaee223",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " It seems like you're trying to describe or represent a list of points in a 2D space, each with an associated distance from some origin. However, the information provided is not easily parsed as a Python list. Here's how you might represent it using NumPy and assuming the first element of each sub-list is the x-coordinate and the second is the y-coordinate:\n",
      "\n",
      "```python\n",
      "import numpy as np\n",
      "points = np.array([[0.43175268, 0.13268092],\n",
      "                   [0.22513067, 0.39157989],\n",
      "                   [-0.19420254, 0.79728609],\n",
      "                   ... # Add the rest of your points here\n",
      "                   ])\n",
      "distances = np.array([373.39196777]) # You should have a corresponding list of distances for each point\n",
      "```\n",
      "\n",
      "This is just one way to represent your data, and you can modify it based on your specific needs. The `numpy.array()` function allows you to create multi-dimensional arrays, where the first dimension's size corresponds to the number of points (or other groups) and the second dimension's size corresponds to the number of components for each point (in this case 2 for x and y).\n",
      "\n",
      "Regarding the \"workspace\" part of your comment, it's unclear from the context what you mean by that. If you're referring to a specific data structure or environment in Python or another programming language, please provide more context so I can give a more accurate answer.\n",
      "\n",
      "CPU times: user 457 ms, sys: 80 ms, total: 537 ms\n",
      "Wall time: 4min 36s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "chain = retrieval | prompt | llm | output_parser\n",
    "\n",
    "for chunk in chain.stream(\"workspace\"):\n",
    "    print(chunk, end=\"\", flush=True)\n",
    "    \n",
    "print(\"\\n\", flush=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dd108c61-edda-41a4-a811-aa2939f0ebd9",
   "metadata": {},
   "source": [
    "## ConversationalRetrievalChain\n",
    "\n",
    "https://python.langchain.com/docs/integrations/providers/vectara/vectara_chat#conversationalretrievalchain-with-question-answering-with-sources"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "32e2f39c-6c4b-4ed4-8aa2-253d9967f6e3",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.chains import LLMChain\n",
    "from langchain.chains import ConversationalRetrievalChain\n",
    "from langchain.chains.conversational_retrieval.prompts import CONDENSE_QUESTION_PROMPT\n",
    "from langchain.chains.question_answering import load_qa_chain\n",
    "from langchain.chains.qa_with_sources import load_qa_with_sources_chain"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "06cd8107-9097-4de1-b3d8-db11609d25bf",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 209 ms, sys: 71.5 ms, total: 281 ms\n",
      "Wall time: 5min 37s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "from langchain.memory import ConversationBufferMemory\n",
    "# We can now create a memory object, which is neccessary to track the inputs/outputs and hold a conversation.\n",
    "memory = ConversationBufferMemory(memory_key=\"chat_history\", return_messages=True)\n",
    "\n",
    "bot = ConversationalRetrievalChain.from_llm(\n",
    "    llm, retriever, memory=memory, verbose=False\n",
    ")\n",
    "\n",
    "result = bot.invoke({\"question\": \"workspace\"})\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "afba9fa6-da38-4ae7-8889-3ea596b3c8c3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "workspace\n"
     ]
    }
   ],
   "source": [
    "print(result[\"question\"], flush=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "1996ef97-6569-45f5-b4d3-3f380f8a2f41",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " The `workspace` keyword in Structurizr DSL is the top level language construct that serves as a wrapper for the `model` and `views`. A workspace can optionally be given a name and description. It also allows extending another workspace to add more elements, relationships, views, etc. Here's the grammar:\n",
      "\n",
      "```\n",
      "workspace [name] [description] {\n",
      "    [extends <file|url>]\n",
      "    ...\n",
      "}\n",
      "\n",
      "workspace extends <file|url> {\n",
      "    // identifiers from base workspace are available here\n",
      "}\n",
      "```\n",
      "\n",
      "Permitted children for `workspace`:\n",
      "- name <name>\n",
      "- description <description>\n",
      "- [properties](#properties)\n",
      "- [!docs](#documentation)\n",
      "- [!adrs](#architecture-decision-records-adrs)\n",
      "- [!identifiers](#identifier-scope)\n",
      "- [!impliedRelationships](#impliedrelationships)\n",
      "- model\n",
      "- views\n",
      "- configuration\n",
      "- [extends <file|url>]\n",
      "\n",
      "The `extends` keyword is optional and allows extending another workspace, which can be a local DSL/JSON file or a remote (via HTTPS URL) DSL/JSON file.\n"
     ]
    }
   ],
   "source": [
    "print(result[\"answer\"], flush=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "711755be-289a-4386-a5d8-0e50c1001062",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Start at    ... 2024-03-12 01:39:39.722983\n",
      "\n",
      "Complete at ... 2024-03-12 01:40:12.446788\n",
      "Total time used 32.72275114059448 seconds\n",
      "\n"
     ]
    }
   ],
   "source": [
    "## Pass in chat history\n",
    "bot = ConversationalRetrievalChain.from_llm(\n",
    "    llm, retriever\n",
    ")\n",
    "\n",
    "time1 = utility.chain_starts()\n",
    "\n",
    "chat_history = []\n",
    "result = bot.invoke({\"question\": query, \"chat_history\": chat_history})\n",
    "\n",
    "utility.chain_completes(time1)\n",
    "\n",
    "# print(result,flush=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "812b2278-93c9-4062-81fa-a126683ed662",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Does company XYZ use structurizr dsl\n"
     ]
    }
   ],
   "source": [
    "print(result[\"question\"], flush=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "848ce8df-70ca-4032-8931-4efab84cfadd",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " Yes, according to the context provided, company XYZ uses Structurizr DSL for software architecture modeling and documentation.\n"
     ]
    }
   ],
   "source": [
    "print(result[\"answer\"], flush=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "d12669ff-6e48-45ed-8d57-72b60e6dd89b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Start at    ... 2024-03-12 01:40:12.476953\n",
      "\n",
      "Complete at ... 2024-03-12 01:40:42.366396\n",
      "Total time used 29.888598203659058 seconds\n",
      "\n"
     ]
    }
   ],
   "source": [
    "## Return Source Documents\n",
    "\n",
    "bot = ConversationalRetrievalChain.from_llm(\n",
    "    llm, retriever, return_source_documents=True\n",
    ")\n",
    "time1 = utility.chain_starts()\n",
    "\n",
    "chat_history = []\n",
    "result = bot.invoke({\"question\": query, \"chat_history\": chat_history})\n",
    "\n",
    "utility.chain_completes(time1)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "0c916586-48b8-432e-b1f1-6dfbd0dd60de",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "answer:\n",
      " Yes, according to the context provided, company XYZ uses Structurizr DSL for software architecture modeling and documentation.\n"
     ]
    }
   ],
   "source": [
    "print(f'answer:\\n{result[\"answer\"]}',flush=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "c5589f93-ce04-47a0-b9d6-96f7d57cca7b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "source_documents:\n",
      "Some famous companies such as ABC and XYZ are using Structurizr DSL for software architecture modeling and documentation.\n"
     ]
    }
   ],
   "source": [
    "print(f'source_documents:\\n{result[\"source_documents\"][0].page_content}',flush=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "85d3057c-1135-46c9-a885-a0d0b8481893",
   "metadata": {},
   "outputs": [],
   "source": [
    "## ConversationalRetrievalChain with map_reduce\n",
    "\n",
    "# question_generator = LLMChain(llm=llm, prompt=CONDENSE_QUESTION_PROMPT)\n",
    "# doc_chain = load_qa_with_sources_chain(llm, chain_type=\"map_reduce\")\n",
    "\n",
    "# chain = ConversationalRetrievalChain(\n",
    "#     retriever=retriever,\n",
    "#     question_generator=question_generator,\n",
    "#     combine_docs_chain=doc_chain,\n",
    "# )\n",
    "\n",
    "# chat_history = []\n",
    "# result = chain.invoke({\"question\": query, \"chat_history\": chat_history})\n",
    "# print(result, flush=True)\n",
    "\n",
    "## ValueError: Document prompt requires documents to have metadata variables: ['source']. Received document with missing metadata: ['source']."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "84dbbaca-a40e-4376-b6cd-8a66b13bc707",
   "metadata": {},
   "outputs": [],
   "source": [
    "## Revise PromptTemplate\n",
    "# from langchain_core.prompts import PromptTemplate\n",
    "\n",
    "# template = \"\"\"Use the following pieces of context to answer the question at the end. \n",
    "# If you don't know the answer, just say that you don't know, don't try to make up an answer. \n",
    "# Use three sentences maximum and keep the answer as concise as possible. \n",
    "# Always say \"thanks for asking!\" at the end of the answer. \n",
    "# {context}\n",
    "# Question: {question}\n",
    "# Helpful Answer:\"\"\"\n",
    "# prompt = PromptTemplate.from_template(template)\n",
    "\n",
    "# chain = (\n",
    "#     {\"context\": retriever, \"question\": RunnablePassthrough()}\n",
    "#     | prompt\n",
    "#     | llm\n",
    "# )\n",
    "\n",
    "# time1 = utility.chain_starts()\n",
    "\n",
    "# result = chain.invoke(query)\n",
    "\n",
    "# print(result, flush=True)\n",
    "\n",
    "# time1 = utility.chain_starts()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bdde19af-7ffa-41d2-9645-b11c2a2b4ef5",
   "metadata": {},
   "source": [
    "## load_qa_with_sources_chain\n",
    "\n",
    "https://api.python.langchain.com/en/latest/chains/langchain.chains.qa_with_sources.loading.load_qa_with_sources_chain.html#langchain.chains.qa_with_sources.loading.load_qa_with_sources_chain"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "cee62dc5-3834-4433-ab65-39879eb48880",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Just for testing purpose to hard code \"source\"=\"abc\" to avoid the error below\n",
    "# ValueError: Document prompt requires documents to have metadata variables: ['source']. Received document with missing metadata: ['source'].\n",
    "docs = retriever.get_relevant_documents(query)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "f3cc9940-71e8-4698-83df-aa0ca6d5e78a",
   "metadata": {},
   "outputs": [],
   "source": [
    "#print(f'{docs[0]}',flush=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "441340ea-962d-4adf-944f-82601cafecdd",
   "metadata": {},
   "outputs": [],
   "source": [
    "for doc in docs:\n",
    "    doc.metadata[\"source\"]=\"abc\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "87c96838-bb4a-4d01-a896-1b72b6d4c606",
   "metadata": {},
   "source": [
    "### run the query first time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "75c82b91-e0a6-4769-a8bf-557b3957f923",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Start at    ... 2024-03-12 02:01:42.715093\n",
      "\n",
      "Complete at ... 2024-03-12 02:06:52.141264\n",
      "Total time used 309.42338728904724 seconds\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from langchain.chains.qa_with_sources import load_qa_with_sources_chain\n",
    "\n",
    "chain = load_qa_with_sources_chain(llm, chain_type=\"map_reduce\", return_intermediate_steps=True)\n",
    "\n",
    "time1 = utility.chain_starts()\n",
    "\n",
    "result = chain.invoke({\"input_documents\": docs, \"question\": query}, return_only_outputs=True)\n",
    "\n",
    "utility.chain_completes(time1)\n",
    "\n",
    "# print(result, flush=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "1ed52b6a-fb66-4481-bd76-06924aa28036",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[' \"Some famous companies such as ABC and XYZ are using Structurizz DSL for software architecture modeling and documentation.\"\\n\\nThis text confirms that Company XYZ is one of the companies mentioned as using Structurizr DSL. The text does not provide additional details about how or to what extent they are using it, but it does establish that they do use it for software architecture modeling and documentation.', ' \"Please see the DSL cookbook for a tutorial guide to the Structurizr DSL.\"\\n\\nThis text suggests that someone has referred to the Structurizr DSL (Domain Specific Language) and provided a resource for learning more about it. However, it does not directly answer whether company XYZ uses the Structurizr DSL or not.']\n"
     ]
    }
   ],
   "source": [
    "print(result[\"intermediate_steps\"], flush=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "3d03c2f4-f019-4adf-9684-c18f73e7c8e5",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " Yes, according to the given sources, Company XYZ uses Structurizr DSL for software architecture modeling and documentation.\n",
      "SOURCES: abc\n"
     ]
    }
   ],
   "source": [
    "print(result[\"output_text\"], flush=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3ed7c59f-ad57-4db2-8429-fbb3bf3354b5",
   "metadata": {},
   "source": [
    "### run the query again to see whether the Answer is consistent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "c1d2f76e-4bb8-4ecc-b31c-e487685a2671",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Start at    ... 2024-03-12 02:06:52.168352\n",
      "\n",
      "Complete at ... 2024-03-12 02:11:44.938654\n",
      "Total time used 292.76894187927246 seconds\n",
      "\n"
     ]
    }
   ],
   "source": [
    "time1 = utility.chain_starts()\n",
    "\n",
    "result = chain.invoke({\"input_documents\": docs, \"question\": query}, return_only_outputs=True)\n",
    "\n",
    "utility.chain_completes(time1)\n",
    "\n",
    "# print(result, flush=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "a9a8d5e7-bba1-4275-b603-92623f2d7ef5",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[' \"Some famous companies such as ABC and XYZ are using Structurizz DSL for software architecture modeling and documentation.\"\\n\\nTherefore, the answer to your question is: Yes, according to the provided text, company XYZ uses Structurizr DSL.', ' \"Please see the DSL cookbook for a tutorial guide to the Structurizr DSL.\"\\n\\nThis text suggests that someone has referred to the Structurizr DSL in this document. However, it does not directly answer the question of whether company XYZ uses Structurizr DSL specifically. To definitively answer the question, additional context or information would be required.']\n"
     ]
    }
   ],
   "source": [
    "print(result[\"intermediate_steps\"], flush=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "98a69a6d-7cd8-46a9-915a-8d5e44d0dace",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " Yes, according to Source \"abc\", company XYZ is mentioned among the famous companies using Structurizr DSL for software architecture modeling and documentation.\n",
      "\n",
      "SOURCES: abc\n"
     ]
    }
   ],
   "source": [
    "print(result[\"output_text\"], flush=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "49c5dd85-9b8d-4e3f-acaa-66b2bb2efe8f",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
